Method and apparatus for obtaining high resolution image, and electronic device for same

ABSTRACT

An image processing method is provided. The image sensor is controlled to output the merged image. It is determined whether there is a target object in the merged image. When there is the target object in the merged image, the merged image is converted into a merged true-color image. An image processing apparatus and an electronic device are provided. With the image processing method, the image processing apparatus and the electronic device, by determining whether there is a target object in the merged image, the image sensor is controlled to output a suitable image. The situation that the image sensor needs to take a long work to output a high quality image can be avoided, such that the work time is reduced, the work efficiency is improved, thus improving the user satisfaction.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority of Chinese PatentApplication No. 201611092363.8, filed on Nov. 29, 2016, the entirecontents of which are incorporated herein by reference.

FIELD

The present disclosure relates to the imaging technology field, and moreparticularly to an image processing method, an image processingapparatus and an electronic device.

BACKGROUND

When an image is processed using a conventional image processing method,either the obtained image has a low resolution, or it takes a long timeand too much resource to obtain an image with high resolution, both ofwhich are inconvenient for users.

DISCLOSURE

The present disclosure aims to solve at least one of existing problemsin the related art to at least extent. Accordingly, the presentdisclosure provides an image processing method, an image processingapparatus and an electronic device.

Embodiments of the present disclosure provide an image processingmethod. The image processing method is applied in an electronic device.The electronic device includes an imaging apparatus including an imagesensor. The image sensor includes an array of photosensitive pixel unitsand an array of filter units arranged on the array of photosensitivepixel units. Each filter unit corresponds to one photosensitive pixelunit, and each photosensitive pixel unit includes a plurality ofphotosensitive pixels. The image processing method includes: outputtinga merged image by the image sensor, in which, the merged image includesan array of combination pixels, and the photosensitive pixels in a samephotosensitive pixel unit are collectively output as one merged pixel;determining whether there is a target object in the merged image; andwhen there is the target object in the merged image, converting themerged image into a merged true-color image.

Embodiments of the present disclosure further provide an imageprocessing apparatus. The image processing apparatus is applied in anelectronic device. The electronic device includes an imaging apparatusincluding an image sensor. The image sensor includes an array ofphotosensitive pixel units and an array of filter units arranged on thearray of photosensitive pixel units. Each filter unit corresponds to onephotosensitive pixel unit, and each photosensitive pixel unit includes aplurality of photosensitive pixels. The image processing apparatusincludes a non-transitory computer-readable medium includingcomputer-readable instructions stored thereon, and an instructionexecution system which is configured by the instructions to implement atleast one of a first control module, a first determining module, and afirst converting module. The first control module is configured tooutput a merged image by the image sensor. The merged image includes anarray of combination pixels, and the photosensitive pixels in a samephotosensitive pixel unit are collectively output as one merged pixel.The first determining module is configured to determine whether there isa target object in the merged image. The first converting module isconfigured to convert the merged image into a merged true-color imagewhen there is the target object in the merged image.

Embodiments of the present disclosure provide an electronic device. Theelectronic device includes a housing, a processor, a memory, a circuitboard, a power supply circuit and an imaging apparatus. The circuitboard is enclosed by the housing. The processor and the memory arepositioned on the circuit board. The power supply circuit is configuredto provide power for respective circuits or components of the electronicdevice. The imaging apparatus includes an image sensor. The image sensorincludes an array of photosensitive pixel units and an array of filterunits arranged on the array of photosensitive pixel units. Each filterunit corresponds to one photosensitive pixel unit, and eachphotosensitive pixel unit includes a plurality of photosensitive pixels.The memory is configured to store executable program codes. Theprocessor is configured to run a program corresponding to the executableprogram codes by reading the executable program codes stored in thememory, to perform the image processing method according to embodimentsof the present disclosure.

Additional aspects and advantages of embodiments of present disclosurewill be given in part in the following descriptions, become apparent inpart from the following descriptions, or be learned from the practice ofthe embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects and advantages of embodiments of the presentdisclosure will become apparent and more readily appreciated from thefollowing descriptions made with reference to the drawings.

FIG. 1 is a flow chart of an image processing method according to anembodiment of the present disclosure.

FIG. 2 is a block diagram of an image sensor according to an embodimentof the present disclosure.

FIG. 3 is a schematic diagram of an image sensor according to anembodiment of the present disclosure.

FIG. 4 is a flow chart of an image processing method according toanother embodiment of the present disclosure.

FIG. 5 is a flow chart of an image processing method according to yetanother embodiment of the present disclosure.

FIG. 6 is a flow chart illustrating a process of converting acolor-block image into a simulation image according to an embodiment ofthe present disclosure.

FIG. 7 is a schematic diagram illustrating a circuit of an image sensoraccording to an embodiment of the present disclosure.

FIG. 8 is a schematic diagram of an array of filter units according toan embodiment of the present disclosure.

FIG. 9 is a schematic diagram of a merged image according to anembodiment of the present disclosure.

FIG. 10 is a schematic diagram of a color-block image according to anembodiment of the present disclosure.

FIG. 11 is a schematic diagram illustrating a process of converting acolor-block image into a simulation image according to an embodiment ofthe present disclosure.

FIG. 12 is a flow chart illustrating a process of converting acolor-block image into a simulation image according to anotherembodiment of the present disclosure.

FIG. 13 is a flow chart illustrating a process of converting acolor-block image into a simulation image according to anotherembodiment of the present disclosure.

FIG. 14 is a schematic diagram showing an image pixel unit of acolor-block image according to an embodiment of the present disclosure.

FIG. 15 is a flow chart illustrating a process of converting acolor-block image into a simulation image according to anotherembodiment of the present disclosure.

FIG. 16 is a flow chart illustrating a process of converting a mergedimage into a merged true-color image according to an embodiment of thepresent disclosure.

FIG. 17 is a block diagram of an image processing apparatus according toan embodiment of the present disclosure.

FIG. 18 is a block diagram of an image processing apparatus according toanother embodiment of the present disclosure.

FIG. 19 is a block diagram of a second converting module according to anembodiment of the present disclosure.

FIG. 20 is a block diagram of a third determining unit in the secondconverting module according to an embodiment of the present disclosure.

FIG. 21 is a block diagram of a second converting module according toanother embodiment of the present disclosure.

FIG. 22 is a block diagram of a second converting module according toanother embodiment of the present disclosure.

FIG. 23 is a block diagram of a first converting module according to anembodiment of the present disclosure.

FIG. 24 is a block diagram of an electronic device 1000 according to anembodiment of the present disclosure.

EMBODIMENTS OF THE PRESENT DISCLOSURE

Reference will now be made in detail to exemplary embodiments, examplesof which are illustrated in the accompanying drawings, in which the sameor similar reference numbers throughout the drawings represent the sameor similar elements or elements having same or similar functions.Embodiments described below with reference to drawings are merelyexemplary and used for explaining the present disclosure, and should notbe understood as limitation to the present disclosure.

In the related art, an image sensor includes an array of photosensitivepixel units and an array of filter units arranged on the array ofphotosensitive pixel unit. Each filter unit corresponds to and coversone photosensitive pixel unit, and each photosensitive pixel unitincludes a plurality of photosensitive pixels. When working, the imagesensor is controlled to output a merged image, which can be convertedinto a merged true-color image by an image processing method and saved.The merged image includes an array of merged pixels, and a plurality ofphotosensitive pixels in a same photosensitive pixel unit arecollectively outputted as one merged pixel. Thus, a signal-to-noiseratio of the merge image is increased. However, a resolution of themerged image is reduced.

Certainly, the image sensor can be controlled to output a high pixelcolor-block image, which includes an array of original pixels, and eachphotosensitive pixel corresponds to one original pixel. However, since aplurality of original pixels corresponding to a same filter unit havethe same color, the resolution of the color-block image still cannot beincreased. Thus, the high pixel color-block image needs to be convertedinto a high pixel simulation image by an interpolation algorithm, inwhich the simulation image includes a Bayer array of simulation pixels.Then, the simulation image can be converted into a simulation true-colorimage by an image processing method and saved. However, theinterpolation algorithm consumes resource and time, and the simulationtrue-color image is not required in all scenes.

Thus, embodiments of the present disclosure provide a novel imageprocessing method.

Referring to FIG. 1, an image processing method is illustrated. Theimage processing method is applied in an electronic device. Theelectronic device includes an imaging apparatus including an imagesensor. As illustrated in FIGS. 2 and 3, the image sensor 10 includes anarray 12 of photosensitive pixel units and an array 14 of filter unitsarranged on the array 12 of photosensitive pixel units. Each filter unit14 a corresponds to one photosensitive pixel unit 12 a, and eachphotosensitive pixel unit 12 a includes a plurality of photosensitivepixels 122. The image processing method includes the followings.

At block 211, the image sensor outputs a merged image.

The merged image includes an array of merged pixels. The photosensitivepixels in a same photosensitive pixel unit are collectively output asone merged pixel.

At block 214, it is determined whether there is a target object in themerged image, if yes, an act at block 216 is executed. The target objectincludes a human face.

At block 216, the merged image is converted into a merged true-colorimage.

In some embodiments, when the user take a photo of a target object, forexample a photo of himself or somebody else, it is unnecessary toacquire an image with too-high quality, such that when the target object(such as a human face) is detected in the merged image, the merged imageis directly converted into the merged true-color image without additionprocessing.

With the image processing method according to embodiments of the presentdisclosure, the image sensor can be controlled to output a suitableimage by determining whether there is a target object (such as a humanface) in the image. In this way, a situation that it takes too much workto output a high quality image by the image sensor can be avoided, thusreducing work time of the electronic device, improving work efficiencyand improving the user experience.

In an embodiment, referring to FIG. 4, based on the embodiment describedwith regard to FIG. 1, when there is no target object in the mergedimage, the image processing method further includes the followings.

At block 217, it is determined whether a brightness of the merged imageis less than or equal to a preset threshold, if yes, the act at block216 is executed. In an embodiment, referring to FIG. 5, based on theembodiment described with regard to FIG. 4, when the brightness of themerged image is greater than the preset threshold, the image processingmethod further includes the followings.

At block 218, the image sensor outputs a color-block image.

The color-block image includes image pixel units arranged in a presetarray. Each image pixel unit includes a plurality of original pixels.Each photosensitive pixel unit corresponds to one image pixel unit, andeach photosensitive pixel corresponds to one original pixel.

At block 220, the color-block image is converted into a simulation imageusing a first interpolation algorithm.

The simulation image includes simulation pixels arranged in an array andeach photosensitive pixel corresponds to one simulation pixel.

At block 230, the simulation image is converted into a simulationtrue-color image.

In some embodiments of the present disclosure, the brightness of themerged image is a key factor for determining whether to output a mergedimage or a color-block image. When the brightness is less than or equalto the preset threshold, the merged image is required; when thebrightness is greater than the preset threshold, the color-block imageis required.

When the brightness is less than or equal to the preset threshold, themerged image is directly converted into the merged true-color imagewithout addition processing.

When the brightness is greater than the preset threshold, the imagesensor is controlled to output the color-block image, and thecolor-block image is converted into the simulation image, then thesimulation image is converted into the simulation true-color image.

In some implementations, a brightness of each merged pixel in the mergedimage may be calculated, or the merged image may be divided into aplurality of brightness analysis areas and then a brightness of eachbrightness analysis area is calculated.

In some implementations, the brightness may be obtained by otherphotosensitive apparatus of the electronic device, and the presentdisclosure is not limited thereto.

In some implementations, the preset threshold for the brightness can beset by a user according to personal preferences. Accordingly, the usercan adopt different shooting modes based on different demands on theimage, thus obtaining an ideal image.

In some implementations, different thresholds for the brightness can bestored in a memory of the electronic device for user's selection.However, the present disclosure is not limited thereto.

Referring to FIG. 6, in some implementations, the act at block 220includes the followings.

At block 221, it is determined whether a color of a simulation pixel isidentical to that of an original pixel at a same position as thesimulation pixel, if yes, an act at block 222 is executed, otherwise, anact at block 223 is executed.

At block 222, a pixel value of the original pixel is determined as apixel value of the simulation pixel.

At block 223, the pixel value of the simulation pixel is determinedaccording to a pixel value of an association pixel.

The association pixel is selected from an image pixel unit with a samecolor as the simulation pixel and adjacent to an image pixel unitincluding the original pixel.

FIG. 7 is a schematic diagram illustrating a circuit of an image sensoraccording to an embodiment of the present disclosure. FIG. 8 is aschematic diagram of an array of filter units according to an embodimentof the present disclosure. FIGS. 2-3 and 7-8 are better viewed together.

Referring to FIGS. 2-3 and 7-8, the image sensor 10 according to anembodiment of the present disclosure includes an array 12 ofphotosensitive pixel units and an array 14 of filter units arranged onthe array 12 of photosensitive pixel units.

Further, the array 12 of photosensitive pixel units includes a pluralityof photosensitive pixel units 12 a. Each photosensitive pixel unit 12 aincludes a plurality of adjacent photosensitive pixels 122. Eachphotosensitive pixel 122 includes a photosensitive element 1222 and atransmission transistor 1224. The photosensitive element 1222 may be aphotodiode, and the transmission transistor 1224 may be a MOStransistor.

The array 14 of filter units includes a plurality of filter units 14 a.Each filter unit 14 a corresponds to one photosensitive pixel unit 12 a.

In detail, in some examples, the filter units are arranged in a Bayerarray. In other words, four adjacent filter units 14 a include one redfilter unit, one blue filter unit and two green filter units.

Each photosensitive pixel unit 12 a corresponds to a filter unit 14 awith a same color. If a photosensitive pixel unit 12 a includes nadjacent photosensitive elements 1222, one filter unit 14 a covers nphotosensitive elements 1222 in one photosensitive pixel unit 12 a. Thefilter unit 14 a may be formed integrally, or may be formed byassembling n separate sub filters.

In some implementations, each photosensitive pixel unit 12 a includesfour adjacent photosensitive pixels 122. Two adjacent photosensitivepixels 122 collectively form one photosensitive pixel subunit 120. Thephotosensitive pixel subunit 120 further includes a source follower 124and an analog-to-digital converter 126. The photosensitive pixel unit 12a further includes an adder 128. A first electrode of each transmissiontransistor 1224 in the photosensitive pixel subunit 120 is coupled to acathode electrode of a corresponding photosensitive element 1222. Secondelectrodes of all the transmission transistors 1224 are collectivelycoupled to a gate electrode of the source follower 124 and coupled to ananalog-to-digital converter 126 via the source electrode of the sourcefollower 124. The source follower 124 may be a MOS transistor. Twophotosensitive pixel subunits 120 are coupled to the adder 128 viarespective source followers 124 and respective analog-to-digitalconverters 126.

In other words, four adjacent photosensitive elements 1222 in onephotosensitive pixel unit 12 a of the image sensor 10 according to anembodiment of the present disclosure collectively use one filter unit 14a with a same color as the photosensitive pixel unit. Eachphotosensitive element 1222 is coupled to a transmission transistor 1224correspondingly. Two adjacent photosensitive elements 1222 collectivelyuse one source follower 124 and one analog-digital converter 126. Fouradjacent photosensitive elements 1222 collectively use one adder 128.

Further, four adjacent photosensitive elements 1222 are arranged in a2-by-2 array. Two photosensitive elements 1222 in one photosensitivepixel subunit 120 can be in a same row.

During an imaging process, when two photosensitive pixel subunits 120 orfour photosensitive elements 1222 covered by a same filter unit 14 a areexposed simultaneously, pixels can be merged, and the merged image canbe outputted.

In detail, the photosensitive element 1222 is configured to convertlight into charge, and the charge is proportional to an illuminationintensity. The transmission transistor 1224 is configured to control acircuit to turn on or off according to a control signal. When thecircuit is turned on, the source follower 124 is configured to convertthe charge generated through light illumination into a voltage signal.The analog-to-digital converter 126 is configured to convert the voltagesignal into a digital signal. The adder 128 is configured to add twodigital signals for outputting.

Referring to FIG. 9, take an image sensor 10 of 16M as an example. Theimage sensor 10 according to an embodiment of the present disclosure canmerge photosensitive pixels 122 of 16M into photosensitive pixels of 4M,i.e., the image sensor 10 outputs the merged image. After the merging,the photosensitive pixel 122 quadruples in size, such that thephotosensibility of the photosensitive pixel 122 is increased. Inaddition, since most part of noise in the image sensor 10 is random,there may be noise points at one or two pixels. After fourphotosensitive pixels 122 are merged into a big photosensitive pixel122, an effect of noise points on the big photosensitive pixel isreduced, i.e., the noise is weakened and SNR (signal to noise ratio) isimproved.

However, when the size of the photosensitive pixel 122 is increased, thepixel value is decreased, and thus the resolution of the merged image isdecreased.

During an imaging process, when four photosensitive elements 1222covered by a same filter unit 14 a are exposed in sequence, acolor-block image is output.

In detail, the photosensitive element 1222 is configured to convertlight into charge, and the charge is proportional to an illuminationintensity. The transmission transistor 1224 is configured to control acircuit to turn on or off according to a control signal. When thecircuit is turned on, the source follower 124 is configured to convertthe charge generated through light illumination into a voltage signal.The analog-to-digital converter 126 is configured to convert the voltagesignal into a digital signal.

Referring to FIG. 10, take an image sensor 10 of 16M as an example. Theimage sensor according to an embodiment of the present disclosure canoutput photosensitive pixels 122 of 16M, i.e., the image sensor 10outputs the color-block image. The color-block image includes imagepixel units. The image pixel unit includes original pixels arranged in a2-by-2 array. The size of the original pixel is the same as that of thephotosensitive pixel 122. However, since a filter unit 14 a coveringfour adjacent photosensitive elements 1222 has a same color (i.e.,although four photosensitive elements 1222 are exposed respectively, thefilter unit 14 a covering the four photosensitive elements has a samecolor), four adjacent original pixels in each image pixel unit of theoutput image have a same color, and thus the resolution of the imagecannot be increased.

The image processing method according to an embodiment of the presentdisclosure is able to process the output color-block image to obtain asimulation image.

In some embodiments, when a merged image is output, four adjacentphotosensitive pixels 122 with the same color can be output as onemerged pixel. Accordingly, four adjacent merged pixels in the mergedimage can be considered as being arranged in a typical Bayer array, andcan be processed directly to output a merged true-color image. When acolor-block image is output, each photosensitive pixel 122 is outputseparately. Since four adjacent photosensitive pixels 122 have a samecolor, four adjacent original pixels in an image pixel unit have a samecolor, which form an untypical Bayer array. However, the untypical Bayerarray cannot be directly processed. In other words, when the imagesensor 10 adopts a same apparatus for processing the image, in order torealize a compatibility of the true-color image outputs under two modes(i.e., the merged true-color image under a merged mode and thesimulation true-color image under a color-block mode), it is required toconvert the color-block image into the simulation image, or to convertthe image pixel unit in an untypical Bayer array into pixels arranged inthe typical Bayer array.

The simulation image includes simulation pixels arranged in the Bayerarray. Each photosensitive pixel corresponds to one simulation pixel.One simulation pixel in the simulation image corresponds to an originalpixel located at the same position as the simulation pixel and in thecolor-block image.

Referring to FIG. 11, for the simulation pixels R3′3′ and R5′5′, thecorresponding original pixels are R33 and B55.

When the simulation pixel R3′3′ is obtained, since the simulation pixelR3′3′ has the same color as the corresponding original pixel R33, thepixel value of the original pixel R33 is directly determined as thepixel value of the simulation pixel R3′3′ during conversion.

When the simulation pixel R5′5′ is obtained, since the simulation pixelR5′5′ has a color different from that of the corresponding originalpixel B55, the pixel value of the original pixel B55 cannot be directlydetermined as the pixel value of the simulation pixel R5′5′, and it isrequired to calculate the pixel value of the simulation pixel R5′5′according to an association pixel of the simulation pixel R5′5′ by aninterpolation algorithm.

It should be noted that, a pixel value of a pixel mentioned in thecontext should be understood in a broad sense as a color attribute valueof the pixel, such as a color value.

There may be more than one association pixel unit for each simulationpixel, for example, there may be four association pixel units, in whichthe association pixel units have the same color as the simulation pixeland are adjacent to the image pixel unit including the original pixel atthe same position as the simulation pixel.

It should be noted that, “adjacent” here should be understood in a broadsense. Take FIG. 11 as an example, the simulation pixel R5′5′corresponds to the original pixel B55. The image pixel units 400, 500,600 and 700 are selected as the association pixel units, but other redimage pixel units far away from the image pixel unit where the originalpixel B55 is located are not selected as the association pixel units. Ineach association pixel unit, the red original pixel closest to theoriginal pixel B55 is selected as the association pixel, which meansthat the association pixels of the simulation pixel R5′5′ include theoriginal pixels R44, R74, R47 and R77. The simulation pixel R5′5′ isadjacent to and has the same color as the original pixels R44, R74, R47and R77.

In different cases, the original pixels can be converted into thesimulation pixels in different ways, thus converting the color-blockimage into the simulation image. Since the filters in the Bayer arrayare adopted when shooting the image, the SNR of the image is improved.During the image processing procedure, the interpolation processing isperformed on the color-block image by the interpolation algorithm, suchthat the distinguishability and resolution of the image can be improved.

Referring to FIG. 12, in some implementations, the act at block 223(i.e., determining the pixel value of the simulation pixel according tothe pixel value of the association pixel) includes the followings.

At block 2232, a change of the color of the simulation pixel in eachdirection of at least two directions is calculated according to thepixel value of the association pixel.

At block 2234, a weight in each direction of the at least two directionsis calculated according to the change.

At block 2236, the pixel value of the simulation pixel is calculatedaccording to the weight and the pixel value of the association pixel.

In detail, the interpolation processing is realized as follows: withreference to energy changes of the image in different directions andaccording to weights of the association pixels in different directions,the pixel value of the simulation pixel is calculated by a linearinterpolation. From the direction having a smaller energy change, it canget a higher reference value, i.e., the weight for this direction in theinterpolation is high.

In some examples, for sake of convenience, only the horizontal directionand the vertical direction are considered.

The pixel value of the simulation pixel R5′5′ is obtained by aninterpolation based on the original pixels R44, R74, R47 and R77. Sincethere is no original pixel with a same color as the simulation pixel(i.e., R) in the horizontal direction and the vertical direction of theoriginal pixel R55 corresponding the simulation pixel R5′5′, a componentof this color (i.e., R) in each of the horizontal direction and thevertical direction is calculated according to the association pixels.The components in the horizontal direction are R45 and R75, thecomponents in the vertical direction are R54 and R57. All the componentscan be calculated according to the original pixels R44, R74, R47 andR77.

In detail, R45=R44*⅔+R47*⅓, R75=⅔*R74+⅓*R77, R54=⅔*R44+⅓*R74,R57=⅔*R47+⅓*R77.

The change of color and the weight in each of the horizontal directionand the vertical direction are calculated respectively. In other words,according to the change of color in each direction, the reference weightin each direction used in the interpolation is determined. The weight inthe direction with a small change is high, while the weight in thedirection with a big change is low. The change in the horizontaldirection is X1=1R45-R751. The change in the vertical direction isX2=1R54-R571, W1=X1/(X1+X2), W2=X2/(X1+X2).

After the above calculation, the pixel value of the simulation pixelR5′5′ can be calculated as R5′5′=(⅔*R45+⅓*R75)*W2+(⅔*R54+⅓*R57)*W1. Itcan be understood that, if X1>X2, then W1>W2. The weight in thehorizontal direction is W2, and the weight in the vertical direction isW1, vice versa.

Accordingly, the pixel value of the simulation pixel can be calculatedby the interpolation algorithm. After the calculations on theassociation pixels, the original pixels can be converted into thesimulation pixels arranged in the typical Bayer array. In other words,four adjacent simulation pixels arranged in the 2-by-2 array include onered simulation pixel, two green simulation pixels and one bluesimulation pixel.

It should be noted that, the interpolation processing is not limited tothe above-mentioned method, in which only the pixel values of pixelswith a same color as the simulation pixel in the vertical direction andthe horizontal direction are considered during calculating the pixelvalue of the simulation pixel. In other embodiments, pixel values ofpixels with other colors can also be considered.

Referring to FIG. 13, in some embodiments, before the act at block 223,the method further includes performing a white-balance compensation onthe color-block image, as illustrated at block 224.

Accordingly, after the act at 223, the method further includesperforming a reverse white-balance compensation on the simulation image,as illustrated at block 225.

In detail, in some examples, when converting the color-block image intothe simulation image, during the interpolation, the red and bluesimulation pixels not only refer to the color weights of original pixelshaving the same color as the simulation pixels, but also refer to thecolor weights of original pixels with the green color. Thus, it isrequired to perform the white-balance compensation before theinterpolation to exclude an effect of the white-balance in theinterpolation calculation. In order to avoid the white-balance of thecolor-block image, it is required to perform the reverse white-balancecompensation after the interpolation according to gain values of thered, green and blue colors in the compensation.

In this way, the effect of the white-balance in the interpolationcalculation can be excluded, and the simulation image obtained after theinterpolation can keep the white-balance of the color-block image.

Referring to FIG. 13 again, in some implementations, before the act atblock 223, the method further includes performing a bad-pointcompensation on the color-block image, as illustrated at block 226.

It can be understood that, limited by the manufacturing process, theremay be bad points in the image sensor 10. The bad point presents a samecolor all the time without varying with the photosensibility, whichaffects quality of the image. In order to ensure an accuracy of theinterpolation and prevent from the effect of the bad points, it isrequired to perform the bad-point compensation before the interpolation.

In detail, during the bad-point compensation, the original pixels aredetected. When an original pixel is detected as the bad point, thebad-point compensation is performed according to pixel values of otheroriginal pixels in the image pixel unit where the original pixel islocated.

In this way, the effect of the bad point on the interpolation can beavoided, thus improving the quality of the image.

Referring to FIG. 13 again, in some implementations, before the act atblock 223, the method includes performing a crosstalk compensation onthe color-block image, as illustrated at block 227.

In detail, four photosensitive pixels 122 in one photosensitive pixelunit 12 a cover the filters with the same color, and the photosensitivepixels 122 have differences in photosensibility, such that fixedspectrum noise may occur in pure-color areas in the simulationtrue-color image outputted after converting the simulation image and thequality of the image may be affected. Therefore, it is required toperform the crosstalk compensation.

In some implementations, compensation parameters can be set by:providing a preset luminous environment, configuring imaging parametersof the imaging apparatus, capturing multi-frame images, processing themulti-frame images to obtain crosstalk compensation parameters, andstoring the crosstalk compensation parameters.

As explained above, in order to perform the crosstalk compensation, itis required to obtain the compensation parameters during themanufacturing process of the image sensor of the imaging apparatus, andto store the parameters related to the crosstalk compensation into thestorage of the imaging apparatus or the electronic device provided withthe imaging apparatus, such as the mobile phone or tablet computer.

The preset luminous environment, for example, may include an LED uniformplate having a color temperature of about 5000K and a brightness ofabout 1000 lux. The imaging parameters may include a gain value, ashutter value and a location of a lens. After setting the relatedparameters, the crosstalk compensation parameters can be obtained.

During the process, multiple color-block images are obtained using thepreset imaging parameters in the preset luminous environment, andcombined into one combination color-block image, such that the effect ofnoise caused by using a single color-block image as reference can bereduced.

Referring to FIG. 14, take the image pixel unit Gr as an example. Theimage pixel unit Gr includes original pixels Gr1, Gr2, Gr3 and Gr4. Thepurpose of the crosstalk compensation is to adjust the photosensitivepixels which may have different photosensibilities to have the samephotosensibility. An average pixel value of the image pixel unit isGr_avg=(Gr1+Gr2+Gr3+Gr4)/4, which represents an average level ofphotosensibilities of the four photosensitive pixels. By configuring theaverage value as a reference value, ratios of Gr1/Gr_avg, Gr2/Gr_avg,Gr3/Gr_avg and Gr4/Gr_avg are calculated. It can be understood that, bycalculating a ratio of the pixel value of each original pixel to theaverage pixel value of the image pixel unit, a deviation between eachoriginal pixel and the reference value can be reflected. Four ratios canbe recorded in a storage of a related device as the compensationparameters, and can be retrieved during the imaging process tocompensate for each original pixel, thus reducing the crosstalk andimproving the quality of the image.

Generally, after setting the crosstalk compensation parameters,verification is performed on the parameters to determine the accuracy ofthe parameters.

During the verification, a color-block image is obtained with the sameluminous environment and same imaging parameters as the preset luminousenvironment and the preset imaging parameters, and the crosstalkcompensation is performed on the color-block image according to thecalculated compensation parameters to calculate compensated Gr′_avg,Gr′1/Gr′_avg, Gr′2/Gr′_avg, Gr′3/Gr′_avg and Gr′4/Gr′_avg. The accuracyof parameters can be determined according to the calculation resultsfrom a macro perspective and a micro perspective. From the microperspective, when a certain original pixel after the compensation stillhas a big deviation which is easy to be sensed by the user after theimaging process, it means that the parameters are not accurate. From themacro perspective, when there are too many original pixels withdeviations after the compensation, the deviations as a whole can besensed by the user even if a single original pixel has a smalldeviation, and in this case, the parameters are also not accurate. Thus,a ratio threshold can be set for the micro perspective, and anotherratio threshold and a number threshold can be set for the macroperspective. In this way, the verification can be performed on thecrosstalk compensation parameters to ensure the accuracy of thecompensation parameters and to reduce the effect of the crosstalk on thequality of the image.

Referring to FIG. 15, in some implementations, after the act at block223, the method further includes performing at least one of a mirrorshape correction, a demosaicking processing, a denoising processing andan edge sharpening processing on the simulation image, as illustrated atblock 228.

It can be understood that, after the color-block image is converted intothe simulation image, the simulation pixels are arranged in the typicalBayer array. The simulation image can be processed, during which, themirror shape correction, the demosaicking processing, the denoisingprocessing and the edge sharpening processing are included, such thatthe simulation true-color image can be obtained and output to the user.

Referring to FIG. 16, in some implementations, the act at block 216includes the followings.

At block 2162, the merged image is converted into a restoration imageusing a second interpolation algorithm.

The restoration image includes restoration pixels arranged in an array,and each photosensitive pixel corresponds to one restoration pixel. Acomplexity of the second interpolation algorithm is less than that ofthe first interpolation algorithm.

At block 2164, the restoration image is converted into the mergedtrue-color image.

In some implementations, the algorithm complexity includes the timecomplexity and the space complexity, and both the time complexity andthe space complexity of the second interpolation algorithm are less thanthose of the first interpolation algorithm. The time complexity isconfigured to measure a time consumed by the algorithm, and the spacecomplexity is configured to measure a storage space consumed by thealgorithm. If the time complexity is small, it indicates that thealgorithm consumes little time. If the space complexity is small, itindicates that the algorithm consumes little storage space. Thus, it isadvantageous to improve calculation speed by using the secondinterpolation algorithm, such that the shooting process is smooth, thusimproving the user experience.

In some implementations, the second interpolation algorithm is used toquadruple the merged image without other complicated calculations, suchthat the restoration image corresponding to the simulation image can beobtained.

It can be understood that, after obtaining the simulation true-colorimage, the denoising processing and the edge sharpening processing areperformed on the simulation true-color image. Thus, the simulationtrue-color image with high quality can be obtained after the processingand output to the user.

In another aspect, the present disclosure also provides an imageprocessing apparatus. FIG. 17 is a block diagram of an image processingapparatus according to an embodiment of the present disclosure.Referring to FIG. 17 and FIGS. 2-3 and 7-8, an image processingapparatus 4000 is illustrated. The image processing apparatus 4000 isapplied in an electronic device. The electronic device includes animaging apparatus including an image sensor 10. As illustrated above,the image sensor 10 includes an array 12 of photosensitive pixel unitsand an array 14 of filter units arranged on the array 12 ofphotosensitive pixel units. Each filter unit 14 a corresponds to onephotosensitive pixel unit 12 a, and each photosensitive pixel unit 12 aincludes a plurality of photosensitive pixels 122. The image processingapparatus 4000 includes a non-transitory computer-readable medium 4600and an instruction execution system 4800. The non-transitorycomputer-readable medium 4600 includes computer-executable instructionsstored thereon. As illustrated in FIG. 17, the non-transitorycomputer-readable medium 4600 includes a plurality of program modules,including a first control module 411, a first determining module 414 anda first converting module 416. The instruction execution system 4800 isconfigured by the instructions stored in the medium 4600 to implementthe program modules.

The first control module 411 is configured to output a merged image bythe image sensor 10. The merged image includes an array of mergedpixels, and the photosensitive pixels 122 in a same photosensitive pixelunit 12 a are collectively outputted as one merged pixel. The firstdetermining module 414 is configured to determine whether there is atarget object in the merged image. The target object includes a humanface. The first converting module 416 is configured to convert themerged image into a merged true-color image when there is the targetobject in the merged image.

In other words, the act at block 211 can be implemented by the firstcontrol module 411. The act at block 214 can be implemented by thedetermining module 414. The act at block 216 can be implemented by thefirst converting module 416.

With the image processing apparatus according to embodiments of thepresent disclosure, the image sensor can be controlled to output asuitable image by determining whether there is a target object in theimage. In this way, a situation that it takes too much work to output ahigh quality image by the image sensor can be avoided, thus reducingwork time of the electronic device, improving work efficiency andimproving the user experience.

FIG. 18 is a block diagram of an image processing apparatus according toan embodiment of the present disclosure. As illustrated in FIG. 18,based on the embodiment described with regard to FIG. 17, thenon-transitory computer-readable medium 4600 further includes a seconddetermining module 417, a second control module 418, a second convertingmodule 420 and a third converting module 430. The instruction executionsystem 4800 is configured by the instructions stored in the medium 4600to implement the program modules.

The second determining module 417 is configured to determine whether abrightness of the merged image is less than or equal to a presetthreshold when there is no target object. The second control module 418is configured to output a color-block image by the image sensor 10 whenthe brightness is greater than the preset threshold. The color-blockimage includes image pixel units arranged in a preset array, and eachimage pixel unit includes a plurality of original pixels. Eachphotosensitive pixel unit 12 a corresponds to one image pixel unit, andeach photosensitive pixel 122 corresponds to one original pixel. Thesecond converting module 420 is configured to convert the color-blockimage into a simulation image using a first interpolation algorithm. Thesimulation image includes simulation pixels arranged in an array andeach photosensitive pixel corresponds to one simulation pixel. The thirdconverting module 430 is configured to convert the simulation image intoa simulation true-color image.

In other words, the act at block 217 can be implemented by the secondcontrol module 417. The act at block 218 can be implemented by thesecond control module 418. The act at block 220 can be implemented bythe second converting module 420. The act at block S230 can beimplemented by the third converting module 430.

Referring to FIG. 19, the second converting module 420 includes a firstdetermining unit 421, a second determining unit 422, and a thirddetermining unit 423. The first determining unit 421 is configured todetermine whether a color of a simulation pixel is identical to that ofan original pixel at a same position as the simulation pixel. The seconddetermining unit 422 is configured to determine a pixel value of theoriginal pixel as a pixel value of the simulation pixel when the colorof the simulation pixel is identical to that of the original pixel atthe same position as the simulation pixel. The third determining unit423 is configured to determine the pixel value of the simulation pixelaccording to pixel values of association pixels when the color of thesimulation pixel is different from that of the original pixel at thesame position as the simulation pixel. The association pixels areselected from an image pixel unit with a same color as the simulationpixel and adjacent to the image pixel unit including the original pixel.

In other words, the act at block 221 can be implemented by the firstdetermining unit 421. The act at block 222 can be implemented by thesecond determining unit 422. The act at block 223 can be implemented bythe third determining unit 423.

Referring to FIG. 20, in some implementations, the third determiningunit 423 includes a first calculating subunit 4232, a second calculatingsubunit 4234 and a third calculating subunit 4236. The act at block 2232can be implemented by the first calculating subunit 4232. The act atblock 2234 can be implemented by the second calculating subunit 4234.The act at block 2236 can be implemented by the third calculatingsubunit 4236. In other words, the first calculating subunit 4232 isconfigured to calculate a change of the color of the simulation pixel ineach direction of at least two directions according to the pixel valueof the association pixel. The second calculating subunit 4234 isconfigured to calculate a weight in each direction of the at least twodirections according to the change. The third calculating subunit 4236is configured to calculate the pixel value of the simulation pixelaccording to the weight and the pixel value of the association pixel.

FIG. 21 is a block diagram of a second converting module according toanother embodiment of the present disclosure. Referring to FIG. 21, insome implementations, the second converting module 420 further includesa first compensating unit 424 and a restoring unit 425. The act at block224 can be implemented by the first compensating unit 424. The act atblock 225 can be implemented by the restoring unit 425. In other words,the first compensating unit 424 is configured to perform a white-balancecompensation on the color-block image. The restoring unit 425 isconfigured to perform a reverse white-balance compensation on thesimulation image.

In some implementations, the second converting module 420 furtherincludes a second compensating unit 426. The act at block 226 can beimplemented by the second compensating unit 426. In other words, thesecond compensating unit 426 is configured to perform a bad-pointcompensation on the color-block image.

In some implementations, the second converting module 420 furtherincludes a third compensating unit 427. The act at block 227 can beimplemented by the third compensating unit 427. In other words, thethird compensating unit 427 is configured to perform a crosstalkcompensation on the color-block image.

FIG. 22 is a block diagram of a second converting module according toanother embodiment of the present disclosure. Referring to FIG. 22, insome implementations, the second converting module 420 includes aprocessing unit 428. The act at block 228 can be implemented by theprocessing unit 428. In other words, the processing unit 428 isconfigured to perform at least one of a mirror shape correction, ademosaicking processing, a denoising processing and an edge sharpeningprocessing on the simulation image.

Referring to FIG. 23, in some implementations, the first convertingmodule 416 includes a first converting unit 4162 and a second convertingunit 4164. The first converting unit 4162 is configured to convert themerged image into a restoration image using a second interpolationalgorithm. The restoration image includes restoration pixels arranged inan array, and each photosensitive pixel corresponds to one restorationpixel. A complexity of the second interpolation algorithm is less thanthat of the first interpolation algorithm. The second converting unit4164 is configured to convert the restoration image into the mergedtrue-color image. In other words, the act at block 2162 is implementedby the first converting unit 4162. The act at block 2164 is implementedby the second converting unit 4164.

The present disclosure also provides an electronic device.

FIG. 24 is a block diagram of an electronic device 1000 according to anembodiment of the present disclosure. Referring to FIG. 24, theelectronic device 1000 of the present disclosure includes a housing1001, a processor 1002, a memory 1003, a circuit board 1006, a powersupply circuit 1007 and an imaging apparatus 100, The circuit board 1006is enclosed by the housing 1001. The processor 1002 and the memory 1003are positioned on the circuit board 1006. The power supply circuit 1007is configured to provide power for respective circuits or components ofthe electronic device 1000. The memory 1003 is configured to storeexecutable program codes. The imaging apparatus 100 includes an imagesensor 10. As illustrated above, the image sensor 10 includes an array12 of photosensitive pixel units and an array 14 of filter unitsarranged on the array 12 of photosensitive pixel units. Each filter unit14 a corresponds to one photosensitive pixel unit 12 a, and eachphotosensitive pixel unit 12 a includes a plurality of photosensitivepixels 122.

The processor 1002 is configured to run a program corresponding to theexecutable program codes by reading the executable program codes storedin the memory 1003, to perform following operations: outputting a mergedimage by the image sensor, in which, the merged image includes an arrayof merged pixels, and the photosensitive pixels in a same photosensitivepixel unit are collectively output as one merged pixel; determiningwhether there is a target object in the merged image; when there is thetarget object in the merged image, converting the merged image into amerged true-color image.

In some implementations, the imaging apparatus includes a front cameraor a real camera (not illustrated in FIG. 24).

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to perform followingoperations: when there is no target object, determining whether abrightness of merged image is less than or equal to a preset threshold;and when the brightness is less than or equal to the preset threshold,converting the merged image into the merged true-color image.

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to perform followingoperations: when the brightness is greater than the preset threshold,outputting a color-block image by the image sensor, in which thecolor-block image comprises image pixel units arranged in a presetarray, each image pixel unit includes a plurality of original pixels,each photosensitive pixel unit corresponds to one image pixel unit, andeach photosensitive pixel corresponds to one original pixel; convertingthe color-block image into a simulation image using a firstinterpolation algorithm, in which he simulation image comprisessimulation pixels arranged in an array, and each photosensitive pixelcorresponds to one simulation pixel; and converting the simulation imageinto a simulation true-color image.

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to convert thecolor-block image into a simulation image using a first interpolationalgorithm by acts of: determining whether a color of a simulation pixelis identical to that of an original pixel at a same position as thesimulation pixel; when the color of the simulation pixel is identical tothat of the original pixel at the same position as the simulation pixel,determining a pixel value of the original pixel as a pixel value of thesimulation pixel; and when the color of the simulation pixel isdifferent from that of the original pixel at the same position as thesimulation pixel, determining the pixel value of the simulation pixelaccording to a pixel value of an association pixel, in which theassociation pixel is selected from an image pixel unit with a same coloras the simulation pixel and adjacent to an image pixel unit includingthe original pixel.

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to determine thepixel value of the simulation pixel according to a pixel value of anassociation pixel by acts of: calculating a change of the color of thesimulation pixel in each direction of at least two directions accordingto the pixel value of the association pixel; calculating a weight ineach direction of the at least two directions according to the change;and calculating the pixel value of the simulation pixel according to theweight and the pixel value of the association pixel.

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to perform followingoperations: performing a white-balance compensation on the color-blockimage; and performing a reverse white-balance compensation on thesimulation image.

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to perform followingoperation: performing at least one of a bad-point compensation and acrosstalk compensation on the color-block image.

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to perform followingoperations: performing at least one of a mirror shape correction, ademosaicking processing, a denoising processing and an edge sharpeningprocessing on the simulation image.

In some implementations, the processor 1002 is configured to run aprogram corresponding to the executable program codes by reading theexecutable program codes stored in the memory 1003, to convert themerged image into a merged true-color image by acts of: converting themerged image into a restoration image using a second interpolationalgorithm, in which the restoration image includes restoration pixelsarranged in an array, each photosensitive pixel corresponds to onerestoration pixel, and a complexity of the second interpolationalgorithm is less than that of the first interpolation algorithm; andconverting the restoration image into the merged true-color image.

In some implementations, the electronic device may be an electronicequipment provided with an imaging apparatus, such as a mobile phone ora tablet computer, which is not limited herein.

The electronic device 1000 may further include an inputting component(not illustrated in FIG. 24). It should be understood that, theinputting component may further include one or more of the followings:an inputting interface, a physical button of the electronic device 1000,a microphone, etc.

It should be understood that, the electronic device 1000 may furtherinclude one or more of the following components (not illustrated in FIG.24): an audio component, an input/output (I/O) interface, a sensorcomponent and a communication component. The audio component isconfigured to output and/or input audio signals, for example, the audiocomponent includes a microphone. The I/O interface is configured toprovide an interface between the processor 1002 and peripheral interfacemodules. The sensor component includes one or more sensors to providestatus assessments of various aspects of the electronic device 1000. Thecommunication component is configured to facilitate communication, wiredor wirelessly, between the electronic device 1000 and other devices.

It is to be understood that phraseology and terminology used herein withreference to device or element orientation (such as, terms like“center”, “longitudinal”, “lateral”, “length”, “width”, “height”, “up”,“down”, “front”, “rear”, “left”, “right”, “vertical”, “horizontal”,“top”, “bottom”, “inside”, “outside”, “clockwise”, “anticlockwise”,“axial”, “radial”, “circumferential”) are only used to simplifydescription of the present invention, and do not indicate or imply thatthe device or element referred to must have or operated in a particularorientation. They cannot be seen as limits to the present disclosure.

Moreover, terms of “first” and “second” are only used for descriptionand cannot be seen as indicating or implying relative importance orindicating or implying the number of the indicated technical features.Thus, the features defined with “first” and “second” may comprise orimply at least one of these features. In the description of the presentdisclosure, “a plurality of” means two or more than two, unlessspecified otherwise.

In the present disclosure, unless specified or limited otherwise, theterms “mounted,” “connected,” “coupled,” “fixed” and the like are usedbroadly, and may be, for example, fixed connections, detachableconnections, or integral connections; may also be mechanical orelectrical connections; may also be direct connections or indirectconnections via intervening structures; may also be inner communicationsof two elements or interactions of two elements, which can be understoodby those skilled in the art according to specific situations.

In the present disclosure, unless specified or limited otherwise, astructure in which a first feature is “on” a second feature may includean embodiment in which the first feature directly contacts the secondfeature, and may also include an embodiment in which the first featureindirectly contacts the second feature via an intermediate medium.Moreover, a structure in which a first feature is “on”, “over” or“above” a second feature may indicate that the first feature is rightabove the second feature or obliquely above the second feature, or justindicate that a horizontal level of the first feature is higher than thesecond feature. A structure in which a first feature is “below”, or“under” a second feature may indicate that the first feature is rightunder the second feature or obliquely under the second feature, or justindicate that a horizontal level of the first feature is lower than thesecond feature.

Various embodiments and examples are provided in the followingdescription to implement different structures of the present disclosure.In order to simplify the present disclosure, certain elements andsettings will be described. However, these elements and settings areonly examples and are not intended to limit the present disclosure. Inaddition, reference numerals may be repeated in different examples inthe disclosure. This repeating is for the purpose of simplification andclarity and does not refer to relations between different embodimentsand/or settings. Furthermore, examples of different processes andmaterials are provided in the present disclosure. However, it would beappreciated by those skilled in the art that other processes and/ormaterials may be also applied.

Reference throughout this specification to “an embodiment,” “someembodiments,” “an example,” “a specific example,” or “some examples,”means that a particular feature, structure, material, or characteristicdescribed in connection with the embodiment or example is included in atleast one embodiment or example of the present disclosure. In thisspecification, exemplary descriptions of aforesaid terms are notnecessarily referring to the same embodiment or example. Furthermore,the particular features, structures, materials, or characteristics maybe combined in any suitable manner in one or more embodiments orexamples. Moreover, those skilled in the art could combine differentembodiments or different characteristics in embodiments or examplesdescribed in the present disclosure.

Any process or method described in a flow chart or described herein inother ways may be understood to include one or more modules, segments orportions of codes of executable instructions for achieving specificlogical functions or steps in the process, and the scope of a preferredembodiment of the present disclosure includes other implementations,wherein the order of execution may differ from that which is depicted ordiscussed, including according to involved function, executingconcurrently or with partial concurrence or in the contrary order toperform the function, which should be understood by those skilled in theart.

The logic and/or step described in other manners herein or shown in theflow chart, for example, a particular sequence table of executableinstructions for realizing the logical function, may be specificallyachieved in any computer readable medium to be used by the instructionexecution system, device or equipment (such as the system based oncomputers, the system comprising processors or other systems capable ofacquiring the instruction from the instruction execution system, deviceand equipment and executing the instruction), or to be used incombination with the instruction execution system, device and equipment.As to the specification, “the computer readable medium” may be anydevice adaptive for including, storing, communicating, propagating ortransferring programs to be used by or in combination with theinstruction execution system, device or equipment. More specificexamples of the computer-readable medium comprise but are not limitedto: an electronic connection (an electronic device) with one or morewires, a portable computer enclosure (a magnetic device), a randomaccess memory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or a flash memory), an optical fiber device anda portable compact disk read-only memory (CDROM). In addition, thecomputer-readable medium may even be a paper or other appropriate mediumcapable of printing programs thereon, this is because, for example, thepaper or other appropriate medium may be optically scanned and thenedited, decrypted or processed with other appropriate methods whennecessary to obtain the programs in an electric manner, and then theprograms may be stored in the computer memories.

It should be understood that each part of the present disclosure may berealized by hardware, software, firmware or their combination. In theabove embodiments, a plurality of steps or methods may be realized bythe software or firmware stored in the memory and executed by theappropriate instruction execution system. For example, if it is realizedby the hardware, likewise in another embodiment, the steps or methodsmay be realized by one or a combination of the following techniquesknown in the art: a discrete logic circuit having a logic gate circuitfor realizing a logic function of a data signal, an application-specificintegrated circuit having an appropriate combination logic gate circuit,a programmable gate array (PGA), a field programmable gate array (FPGA),etc.

Those skilled in the art shall understand that all or parts of the stepsin the above exemplifying method for the present disclosure may beachieved by commanding the related hardware with programs, the programsmay be stored in a computer-readable storage medium, and the programscomprise one or a combination of the steps in the method embodiments ofthe present disclosure when running on a computer.

In addition, each function cell of the embodiments of the presentdisclosure may be integrated in a processing module, or these cells maybe separate physical existence, or two or more cells are integrated in aprocessing module. The integrated module may be realized in a form ofhardware or in a form of software function modules. When the integratedmodule is realized in a form of software function module and is sold orused as a standalone product, the integrated module may be stored in acomputer-readable storage medium.

The storage medium mentioned above may be read-only memories, magneticdisks, CD, etc.

Although embodiments of present disclosure have been shown and describedabove, it should be understood that above embodiments are justexplanatory, and cannot be construed to limit the present disclosure,for those skilled in the art, changes, alternatives, and modificationscan be made to the embodiments without departing from spirit, principlesand scope of the present disclosure.

What is claimed is:
 1. An image processing method, applied in anelectronic device, wherein the electronic device comprises an imagingapparatus comprising an image sensor, the image sensor comprises anarray of photosensitive pixel units and an array of filter unitsarranged on the array of photosensitive pixel units, each filter unitcorresponds to one photosensitive pixel unit, and each photosensitivepixel unit comprises a plurality of photosensitive pixels, the imageprocessing method comprises: outputting a merged image by the imagesensor, wherein, the merged image comprises an array of merged pixels,and the photosensitive pixels in a same photosensitive pixel unit arecollectively output as one merged pixel; determining whether there is atarget object in the merged image; and when there is the target objectin the merged image, converting the merged image into a restorationimage using a second interpolation algorithm, wherein the restorationimage comprises restoration pixels arranged in an array, eachphotosensitive pixel corresponds to one restoration pixel; andconverting the restoration image into a merged true-color image.
 2. Theimage processing method according to claim 1, further comprising: whenthere is no target object, determining whether a brightness of mergedimage is less than or equal to a preset threshold; and when thebrightness is less than or equal to the preset threshold, converting themerged image into the merged true-color image.
 3. The image processingmethod according to claim 2, further comprising: when the brightness isgreater than the preset threshold, outputting a color-block image by theimage sensor, wherein, the color-block image comprises image pixel unitsarranged in a preset array, each image pixel unit comprises a pluralityof original pixels, each photosensitive pixel unit corresponds to oneimage pixel unit, and each photosensitive pixel corresponds to oneoriginal pixel; converting the color-block image into a simulation imageusing a first interpolation algorithm, wherein, the simulation imagecomprises simulation pixels arranged in an array, and eachphotosensitive pixel corresponds to one simulation pixel; and convertingthe simulation image into a simulation true-color image.
 4. The imageprocessing method according to claim 3, wherein converting thecolor-block image into a simulation image using a first interpolationalgorithm comprises: determining whether a color of a simulation pixelis identical to that of an original pixel at a same position as thesimulation pixel; when the color of the simulation pixel is identical tothat of the original pixel at the same position as the simulation pixel,determining a pixel value of the original pixel as a pixel value of thesimulation pixel; and when the color of the simulation pixel isdifferent from that of the original pixel at the same position as thesimulation pixel, determining the pixel value of the simulation pixelaccording to a pixel value of an association pixel, wherein theassociation pixel is selected from an image pixel unit with a same coloras the simulation pixel and adjacent to an image pixel unit comprisingthe original pixel.
 5. The image processing method according to claim 3,wherein determining the pixel value of the simulation pixel according toa pixel value of an association pixel comprises: calculating a change ofthe color of the simulation pixel in each direction of at least twodirections according to the pixel value of the association pixel;calculating a weight in each direction of the at least two directionsaccording to the change; and calculating the pixel value of thesimulation pixel according to the weight and the pixel value of theassociation pixel.
 6. The image processing method according to claim 3,further comprising: performing a white-balance compensation on thecolor-block image; and performing a reverse white-balance compensationon the simulation image.
 7. The image processing method according toclaim 3, further comprising: performing at least one of a bad-pointcompensation and a crosstalk compensation on the color-block image. 8.The image processing method according to claim 3, further comprising:performing at least one of a mirror shape correction, a demosaickingprocessing, a denoising processing and an edge sharpening processing onthe simulation image.
 9. The image processing method according to claim3, wherein a complexity of the second interpolation algorithm is lessthan that of the first interpolation algorithm.
 10. An image processingapparatus, applied in an electronic device, wherein the electronicdevice comprises an imaging apparatus comprising an image sensor, theimage sensor comprises an array of photosensitive pixel units and anarray of filter units arranged on the array of photosensitive pixelunits, each filter unit corresponds to one photosensitive pixel unit,and each photosensitive pixel unit comprises a plurality ofphotosensitive pixels; the image processing apparatus comprises anon-transitory computer-readable medium comprising computer-executableinstructions stored thereon, and an instruction execution system whichis configured by the instructions to implement: outputting a mergedimage by the image sensor, wherein, the merged image comprises an arrayof merged pixels, the photosensitive pixels in a same photosensitivepixel unit are collectively output as one merged pixel; determiningwhether there is a target object in the merged image; and converting themerged image into a restoration image using a second interpolationalgorithm, wherein the restoration image comprises restoration pixelsarranged in an array, each photosensitive pixel corresponds to onerestoration pixel; and converting the restoration image into a mergedtrue-color image when there is a target object in the merged image. 11.The image processing apparatus according to claim 10, wherein theinstruction execution system is further configured by the instructionsto implement at least one of: determining whether a brightness of themerged image is less than or equal to a preset threshold when there isno target object; and converting the merged image into the mergedtrue-color image when the brightness is less than or equal to the presetthreshold.
 12. The image processing apparatus according to claim 11,wherein the instruction execution system is further configured by theinstructions to implement at least one of: outputting a color-blockimage by the image sensor when the brightness is greater than the presetthreshold, wherein, the color-block image comprises image pixel unitsarranged in a preset array, each image pixel unit comprises a pluralityof original pixels, each photosensitive pixel unit corresponds to oneimage pixel unit, and each photosensitive pixel corresponds to oneoriginal pixel; converting the color-block image into a simulation imageusing a first interpolation algorithm, wherein, the simulation imagecomprises simulation pixels arranged in an array and each photosensitivepixel corresponds to one simulation pixel; and converting the simulationimage into a simulation true-color image.
 13. The image processingapparatus according to claim 12, wherein the instruction executionsystem is further configured by the instructions to implement at leastone of: a determining whether a color of a simulation pixel is identicalto that of an original pixel at a same position as the simulation pixel;determining a pixel value of the original pixel as a pixel value of thesimulation pixel when the color of the simulation pixel is identical tothat of the original pixel at the same position as the simulation pixel;and determining the pixel value of the simulation pixel according to apixel value of an association pixel when the color of the simulationpixel is different from that of the original pixel at the same positionas the simulation pixel, wherein the association pixel is selected froman image pixel unit with a same color as the simulation pixel andadjacent to an image pixel unit comprising the original pixel.
 14. Theimage processing apparatus according to claim 12, wherein theinstruction execution system is further configured by the instructionsto implement at least one of: calculating a change of the color of thesimulation pixel in each direction of at least two directions accordingto the pixel value of the association pixel; calculating a weight ineach direction of the at least two directions according to the change;and calculating the pixel value of the simulation pixel according to theweight and the pixel value of the association pixel.
 15. The imageprocessing apparatus according to claim 12, wherein the preset arraycomprises a Bayer array.
 16. The image processing apparatus according toclaim 12, wherein the image pixel unit comprises original pixelsarranged in a 2-by-2 array.
 17. The image processing apparatus accordingto claim 12, wherein the instruction execution system is furtherconfigured by the instructions to implement at least one of: performinga white-balance compensation on the color-block image; and performing areverse white-balance compensation on the simulation image.
 18. Theimage processing apparatus according to claim 12, wherein theinstruction execution system is further configured by the instructionsto implement at least one of performing a bad-point compensation on thecolor-block image; and performing a crosstalk compensation on thecolor-block image.
 19. The image processing apparatus according to claim12, wherein the instruction execution system is further configured bythe instructions to implement: performing at least one of a mirror shapecorrection, a demosaicking processing, a denoising processing and anedge sharpening processing on the simulation image.
 20. An electronicdevice, comprising a housing, a processor, a memory, a circuit board, apower supply circuit, and an imaging apparatus, wherein, the circuitboard is enclosed by the housing; the processor and the memory arepositioned on the circuit board; the power supply circuit is configuredto provide power for respective circuits or components of the electronicdevice; the imaging apparatus comprises an image sensor, wherein theimage sensor comprises an array of photosensitive pixel units and anarray of filter units arranged on the array of photosensitive pixelunits, each filter unit corresponds to one photosensitive pixel unit,and each photosensitive pixel unit comprises a plurality ofphotosensitive pixels; the memory is configured to store executableprogram codes; and the processor is configured to run a programcorresponding to the executable program codes by reading the executableprogram codes stored in the memory, to perform following operations:outputting a merged image by the image sensor, wherein, the merged imagecomprises an array of merged pixels, and the photosensitive pixels in asame photosensitive pixel unit are collectively output as one mergedpixel; determining whether there is a target object in the merged image;and when there is the target object in the merged image, converting themerged image into a restoration image using a second interpolationalgorithm, wherein the restoration image comprises restoration pixelsarranged in an array, each photosensitive pixel corresponds to onerestoration pixel; and converting the restoration image into a mergedtrue-color image.